Link throughput enhancer

ABSTRACT

An apparatus and communication method that provide for communicating information on a standardized link compliant with a published standard and communicating information on a private link capable of operation at a throughput higher than maximum throughput of the published standard. The apparatus and method further provide for monitoring private link performance and increasing information throughput on the private link above the maximum standard throughput by an amount determined by the monitored private link performance.

BACKGROUND

Performance improvements in computing and storage, along with a drive toexploit these improvements in highly challenging applications, haveincreased the demand for extremely fast data links, for example in areasof high-speed and data-intensive networking. One example of a highlychallenging application is data replication in information storage andretrieval. For systems that are expected to operate continuously, aduplicate and fully operational backup capability is implemented in theevent a primary system fails.

Data replication, which may also be called mirroring, is a process thatgenerates a mirrored copy of information on two or more storage devices,a primary volume and one or more secondary volumes. The goal isavailability of important information on duplicate storage devices atall times. The duplicates may reside on the same or different devices orsystems. Similarly, the duplicates may reside on local or remote devicesor systems. The obvious advantage of remote replication is avoidance ofdestruction of both the primary and secondary copies in the event of adisaster occurring in one location.

Corporations, institutions, and agencies sharing common databases andstorage systems often include enterprise units that are widely dispersedgeographically and therefore may use data replication over very largedistances. Distance between storage sites increases communicationlatency, and reduces speed and reliability, although the demand for fastcommunication remains.

In response to the demand for fast data links, various networkinterconnect standards have been developed to enable fastercommunication between computers and input/output devices. One example ofan interconnect standard is a Fibre Channel (FC) standard and associatedvariants, which are defined in an effort to facilitate datacommunication, including network and channel communication, between andamong multiple processors and peripheral devices. The Fibre Channelstandard enables transfers of large information amounts at very highrates of two or more gigabits (Gb) per second.

Remote replication links in storage systems tend to be exclusivelystandard links with a specified standard throughput, for example 1-2 Gbfor the Fibre Channel standard. Replication links may be implemented onother standards, such as Enterprise Systems Connection (ESCON), SmallComputer Systems Interface (SCSI), and others.

SUMMARY

A communication method comprises communicating information on astandardized link compliant with a published standard and communicatinginformation on a private link capable of operation at a throughputhigher than maximum throughput of the published standard. The methodfurther comprises monitoring private link performance and increasinginformation throughput on the private link above the maximum standardthroughput by an amount determined by the monitored private linkperformance.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention relating to both structure and method ofoperation, may best be understood by referring to the followingdescription and accompanying drawings:

FIG. 1 is a schematic block diagram illustrating an embodiment of astorage apparatus configured to perform private link throughputenhancement;

FIG. 2 is a schematic block diagram depicting an embodiment of a storageapparatus that implements a multiple-tiered technique to increasecommunication link throughput;

FIG. 3A is a flow chart showing an embodiment of a method of compactingdata for serial transmission in an optical network;

FIG. 3B is a time graph illustrating operation of the embodiment of thedata compaction method; and

FIG. 4 is a schematic block diagram illustrating an embodiment of astorage apparatus that implements a technique using bit error ratemeasurements for enhancing throughput on a communication link.

DETAILED DESCRIPTION

For a network or interconnect system or for a geographically dispersedstorage or database system that implements industry standard components,devices, and other infrastructure elements, communication rates arelimited to the throughput specified by the standard. In variousapplications, configurations, and conditions, the communication linksconnecting particular networked devices may have a length much less thanthe maximum length defined by the standard. Similarly, the communicationlinks may have a signal quality much higher than the minimum specifiedfor the standard.

An interconnect structure or network may include one or more nonstandardlinks and exploit the nonstandard character of the link by disregardingconstraints to operate according to industry standard throughput rates,although the communication may follow protocols similar or identical toindustry standard protocols. By disregarding the data rate constraints,an illustrative system can substantially increase throughput. In variousembodiments, the nonstandard links may be private links, dark fiberlinks, dark fiber point-to-point links, or other links.

Standards are set according to attainable technological capabilities intypical operating conditions and configurations. Standards forcommunication links are accordingly set to handle communications betweensites separated by hundreds of miles with limited signal quality. Thedisclosed structures and communication techniques acknowledge that many,in fact most, communication links are substantially shorter and havesubstantially higher signal quality than the “least common denominator”to which the standards are set. The disclosed structures andcommunications techniques exploit the expected more favorablecommunication conditions, when appropriate, to enhance throughput.Accordingly, the disclosed structures and communication techniquesgenerally produce a significant throughput improvement. In manyembodiments, the disclosed structures and communication techniquesinclude support for the standard techniques so that throughput in thesystem disclosed herein is at least as good as the standardimplementation.

Referring to FIG. 1, a schematic block diagram illustrates an embodimentof a storage apparatus 100 configured to perform private link throughputenhancement. The apparatus 100 comprises a communication controller 102adapted to communicate information on a standardized link 108 that iscompliant with a published standard, and on a private link 104 capableof operation at a throughput higher than maximum throughput of thepublished standard. The communication controller 102 can monitor privatelink performance and selectively increase information throughput on theprivate link 104 above the maximum standard throughput based on linkperformance.

A private link 104 is defined herein as a communication link that isexclusive of industry standard links that operate at an industrystandard throughput rate. A common example of an industry standardcommunication link is the Fibre Channel standard. In some embodiments,the private link 104 may be further defined as a point-to-pointcommunication link between network devices, such as storage arrays 106.In various embodiments, a private link 104 may be further defined as acommunication link that does not have one terminus directly attached toa public, shared network which supports a standard managed network.Although a signal may transit a private link and ultimately reach astandard network link, the private link retains character as a privatelink. In some embodiments, private links 104 may be constructed withdedicated leased lines, dial-up lines, satellite links, microwave links,or the like. The private links 104 may connect local sites or remotesites. Private links 104 are typically considered private on the basisthat no other traffic than traffic of the entity leasing or controllingthe private links communicates on the links.

Private networks have traditionally been built with dedicated leasedlines, dial-up lines, or other links such as satellite or microwave.Links are established among remote sites. The links are “private”because no other traffic except the traffic of the company leasing thelinks crosses the links

In a typical embodiment, the communication controller 102 is adapted toreside within storage arrays and communicate information on apoint-to-point remote replication link 108 interconnecting one or morestorage arrays 106. In some embodiments, the communication controller102 manages information communication information on a private darkfiber point-to-point link connecting remotely replicated storage arrays106. In some embodiments, a point-to-point link may use aspecially-modified Dense Wavelength Division Multiplexing (DWDM)extender which has a capability to adjust to varying bit times.

Dark fiber is defined as an optical fiber infrastructure, includingcables and repeaters that are currently installed and existing, but isnot currently used in a public network. A sometimes substantial amountof dark fiber-optic cable may exist because the cost of installing cablein excess of expected short-term capacity is small in comparison toinstalling more cable at a later time. The dark fiber may be leased forusage by individuals or small operations and thus may be neithercontrolled nor connected to a public utility. Accordingly, the smalloperation or even an individual may supply components sufficient to makethe dark fiber operational. The dark fiber may further be used tointerconnect relatively local components of a storage system.

In various embodiments, the storage apparatus 100 is an adapter, device,component, or the like that facilitates and enhances communication.Common examples of the storage apparatus 100 are Network Interface Cards(NICs), Transmission Control Protocol/Internet Protocol (TCP-IP) NICs,Host Bus Adapters (HBAs), Storage Area Network (SAN) HBAs, networkmanagement appliances, intelligent switches, or any similar functionalelement. The storage apparatus 100 is commonly used to offloadinput/output operations from a computing element such as a host computerso that data transfer rates increase. Although TCP-IP Network InterfaceCards (NICs) and SAN Host Bus Adapters (HBAs) vary greatly in the amountof attainable host offloading, a generally common characteristic ofstandard input/output interfaces, regardless of communication protocol,is that a maximum theoretical distance, and corresponding minimumper-bit clock rate, is de-rated by as much as 20-100% to account forwide variability in link type, link quality, fiber bend radius,cross-talk, and poor mechanical connections.

The effect of the lowering the rated optical transmission capability dueto such deterioration or insufficiency is that the published interfacestandard is highly compromised to account for compatibility under worstcase or least common denominator conditions. For example, the interfacestandard of 2 gigabits (Gb) per second for Fibre Channel may, underbest-case conditions, easily enable a throughput of two to ten times thebit rate of the published specification.

Under industry standard constraints, a private link, withoutintermediate commercial repeaters or switches, is commonly used as theweakest link in what may be a multiple million dollar enterprise datareplication scheme. The cost of leased dark fibers over time mayoverwhelm the original hardware cost. Accordingly, the illustrativeapparatus 100 and associated operating method may enable a tremendouscompetitive advantage, possibly a cost advantage in the range ofmillions of dollars, over several years.

In some embodiments, the communication controller 102 may be adapted tocommunicate information on the private link 104 using a communicationtechnique selected from among multiple communication techniques that areimplemented and operable on the private link 104. In a typicalembodiment, the communication controller 102 supports communicationtechniques operable on the private link 104 including an industrystandard communications technique and an increased throughputcommunications technique.

In a particular embodiment, the apparatus 100 may further comprise a biterror rate analyzer 110 coupled to the communication controller 102. Thecommunication controller 102 can selectively communicate information onthe private link 104 using a communication technique that increasesthroughput rate while balancing bit rate and bit error rate.

In some embodiments, the communication controller 102 is adapted tocommunicate information on the private link 104 using a digital pulsewidth modulation communication technique that encodes information as anincrement in pulse width over a nominal pulse width.

Other embodiments may combine multiple techniques, for example usingboth the bit rate analyzer 110 and digital pulse width modulation.Accordingly, throughput is increased by selectively communicatinginformation on the private link 104 by balancing bit rate and bit errorrate, and throughput is further enhanced by the digital pulse widthmodulation technique.

The particular communication technique that executes at a particulartime can be determined using various techniques. In some conditions,implementations, and/or embodiments, the apparatus 100 includes a linkanalyzer 112 that is connected to the private link 104 and operates inconjunction with the communication controller 102. The link analyzer 112determines a condition of communications on the private link 104. Thecommunication controller 102 selects from among the multiplecommunication techniques based on the communications condition.

In some conditions, implementations, and/or embodiments, the apparatus100 further comprises a graphical user interface 114 that interacts withthe communication controller 102 and is operated by a user to selectfrom among the multiple communication techniques.

Referring to FIG. 2, a schematic block diagram depicts an embodiment ofa storage apparatus 200 that implements a multiple-tiered technique toincrease communication link throughput. The storage apparatus 200 may beimplemented in any suitable electronic system, device, or component suchas, for example, a host bus adapter, a storage controller, a diskcontroller, a network management appliance, or others. The storageapparatus 200 comprises a communication controller 202 that communicatesinformation on a communication link 204 using a communication techniqueselected from among multiple-tiered, progressively higher throughputcommunication techniques. The communication controller 202 monitorserror rate on the communication link 204 and selects the communicationtechnique and information throughput based on the monitored error rate.

In various embodiments and/or in particular conditions, the selectionmay be made either under external control or automatically. For examplein some embodiments, a graphical user interface (GUI) may be implementedto enable selection of an operational communication technique from amongthe techniques of the multiple implemented tiers. In some embodiments,the communication technique may be automatically determined according toresults of information sensed as the link communicates information. Someembodiments may include both the GUI support and sensors that enableautomatic communication technique selection.

In a particular embodiment, a Tier-0 technique may enable usage of thestandard interface, for example a Fibre Channel interface, withoutalteration. The communication controller 202, based on selection of acommunication technique, activates the communication technique selectedfrom among the industry standard communications technique and one of theincreased-throughput communication techniques.

In an example configuration, the Tier-0 technique may activate anindustry-standard host bus adapter (HBA) and possibly industry-standardswitches and/or repeaters without modification so that the bit ratenative to the industry-standard components is implemented without ratemodification. The apparatus 200 includes a graphical user interface(GUI) 206 and a throughput control logic 208, for example residing on athroughput control-enabled host bus adapter 200. During high-throughputoperation with non-standard usage of the protocol's bit rate active, forexample with either Tier-1, Tier-2, or Tier-3 operating, a user mayrevert back to Tier-0 operation, thereby changing communication bit rateto the standard rate.

An embodiment may implement Tier-1 as a logic that enables a decrease in1/0 signal bit time by increasing the bit clock rate, for example by anincrease of one to ten times the standard bit clock rate, for linkswhich are shorter and/or having higher signal quality than a worst casecondition.

Some implementations may configure Tier-2 to include a suitable type ofsignal data compaction. A typical throughput increase resulting fromTier-2 operation may be in a range of four, eight, sixteen or more timesnormal throughput in an illustrative example whereby the communicationcontroller 202 uses digital pulse width modulation to encode informationas an increment in pulse width over a nominal pulse width. Digital pulsewidth modulation can be implemented in a manner that stretches the bittime and interprets the increase in width as information content of fourbits (a nibble), a byte (eight bits), a word (sixteen bits), or more.

In a general digital pulse width modulation application, a pulse with adefined nominal pulse width is extended an interval based on a digitalvalue to be communicated. The number of secondary clock cycles that theextended pulse width exceeds the nominal pulse width is counted. Thecount corresponds to the digital value. Various embodiments use acounter to determine the extent that the modulated pulse width exceedsthe nominal pulse width.

Pulse width encodes the digital value communicated and counters are usedto extract the digital data. The digital pulse width modulator does notuse a carrier but instead uses synchronized transmitter and receiverclocks running at substantially the same frequency so that the amount oftime the modulated pulse is extended is measurable. The digital pulsewidth modulator performs data compaction by representing either agreater number of bits in the same time frame or the same number of bitsin a smaller reference frame.

The illustrative storage apparatus 200 is depicted in the form of astorage controller or possibly a host bus adapter, The apparatus mayinclude a plurality of embedded processors 210 generally arehigh-performance processors that are capable of transferring informationat a high rate to support multiple storage devices in a scaleablestorage array controller. A memory controller 212 may be connected tothe embedded processors 210 and operates as a hub device to transferdata point-to-point or, in some embodiments on a network fabric, amongthe multiple storage levels. The illustrative memory controller 212 hasmultiple channels for communicating with a cache memory 222 to ensuresufficient bandwidth for data caching and program execution. The memorycontroller 212 has sufficient performance to manage the multiple I/Ochannels 218.

An Ethernet interface 216 communicates with the memory controller 212via the communication controller 202 which may be an input/output (I/O)controller hub that includes an integrated Fast Ethernet Media AccessController (MAC) to form a local area network (LAN) management interfaceport. The I/O controller hub 214 includes typical peripheral interfacesincluding Universal Serial Bus (USB), Peripheral Component Interconnect(PCI), Integrated Drive Electronics (IDE), General Purpose Input/Output(GPIO), System Management Bus (SMBus), and the like.

The link 204 may connect to a switch fabric interconnect device 220 anda controller such as a Gigabit Ethernet or Fibre Channel controllerbased upon the type of network fabric, iSCSI or Fibre Channel.

Referring to FIG. 3A, a flow chart depicts an embodiment of a method ofcompacting data 300 for serial transmission in an optical network. Inaction 302, digital data is encoded in a pulse width modulated opticalsignal. The pulse width modulation optical signal is transmitted inaction 304. On the receiver side, the pulse width modulator opticalsignal is detected in action 306. The number of clock cycles that thedigital pulse width modulator pulse exceeds a nominal pulse width iscounted in action 308. The count corresponds to the length of a pulseand the value of the digital data communicated.

Referring to FIG. 3B, a time graph 310 illustrates the operation of theembodiment of the data compaction method. The time graph 310 showsencoding of digital data through modulation of pulse width. Anunmodulated pulse train waveform 312 has two states, “one” or “zero”.Each “one” is indicated by a pulse of nominal width 314. The duration ofthe pulse is termed “one time” or “one count”. Time duration between thepulses is called “zero time” or “zero count”.

Digital data is communicated by modulating the pulse train waveform 312to produce a pulse width modulated waveform 316. The modulationeffectively shifts the falling edge of the modulated pulse.

In one embodiment, the nominal pulse width is only increased, notdecreased, to encode the digital information. The pulse width isincreased only in discrete amounts such that the pulse falling edge mayoccur at n possible locations including the position of the falling edgeof the nominal pulse. In one embodiment, the maximum modulated pulsewidth is twice the nominal pulse width.

The pulse rising edge indicates transmission of a new digital datavalue. The pulse falling edge indicates the end of transmission of thedigital data value and is determinative of the value. The data value maybe recovered by determining the amount the modulated pulse width exceedsthe nominal pulse width. In one embodiment, a demodulation counterdetermines the data value. The counter effectively functions as adigital integrator but is unaffected by amplitude, phase, or frequencyof the modulated signal, other than triggering thresholds that start andstop the counter. The digital value is indicated only by the change innominal width, rather than the total pulse width. Therefore the counteris to account for the nominal pulse width. One embodiment accounts forthe nominal pulse width by integrating or counting throughout the entirepulse width and subtracting the nominal pulse width count. Anotherembodiment resets the counter or configures the counter to rollover atthe count associated with the nominal pulse width count.

A Tier-3 implementation may combine Tier-1 and Tier-2 concurrently toattain a shared effect in the illustrative example of four to eightytimes the standard throughput. Tier-3 is typically effective for linksthat are much shorter and/or having much higher signal quality than thelimits imposed by the standard. Typically, conditions enabling Tier-3operation are attainable for links that are confined to a singleelectronics system rack or to a single data center. The Graphical UserInterface (GUI) 206 may be used to activate Tier-3 operation based, forexample, on knowledge of link length and conditions. The GUI 206 mayalso access and display information from an attached bit rate tester orother link analyzer to enable the user to determine whether Tier-3operation is appropriate.

In some embodiments, Tier-3 may include dynamically-selected compactionwhereby the increase in throughput rate of Tier-2 data compaction isselected according to line quality, as determined by the bit-error-ratemeasured in Tier-1 operation.

Referring to FIG. 4, a schematic block diagram illustrates an embodimentof a storage apparatus 400 that implements a technique using bit errorrate measurements for enhancing throughput on a communication link 404.The storage apparatus 400 comprises a bit-error-rate analyzer 406 and acommunication controller 402 that function in combination to communicateinformation on the communication link 404 using a communicationtechnique that selectively increases clock rate throughput and mutuallyoptimizes bit rate and bit error rate, increasing throughput rate whilebalancing bit rate and bit error rate.

The communication controller 402 may increase the bit rate, for examplein a range from one to ten times the standard rate, for a communicationlink that is somewhat shorter and higher quality than the worst caselink under the industry standard. An example of a higher qualitycommunication link may be a combined optical/copper cable with arelatively small number of mechanical connectors or an optical cablewith small radius bends, or a copper cable.

In an implementation example, a graphical user interface (GUI) 408 on ahost computer may be operational, following connection of host busadapters and communication links in operational positions, in a linkthroughput test mode. The GUI 408 may include a slider bar which enablesoptimization of either bit-rate or bit-error-rate in two differentoperating modes.

In a first mode, a user can select a GUI setting to enable afree-floating bit-error-rate and a fixed bit-rate. The user sets theslider bar to an arbitrary bit-rate setting within an allowablethroughput enhancement range. A typical enhancement range may be one toten times the standard throughput, although any suitable enhancement maybe implemented. The bit-error-rate analyzer 406 continuously runs andreports a real-time bit-error-rate. For example, a sample test patternmay be transmitted back and forth across the link or links in abi-directional manner while the bit-error-rate is calculated inreal-time. The slider bar may be left at a maximum bit rate at which anacceptable bit-rate-error can be empirically attained. For example, onebit error in 10¹⁰ bits may be defined as a low standard. One bit errorin 10¹⁵ bits may be specified as an average standard. One bit error in10¹⁷ bits may be assigned to a high standard. The time to send such alarge number of test bits over a link is too lengthy to be feasible. Amore realistic and typical empirical approach is to continually reducethe bit time until a first error occurs, then reduce the bit-rate by ade-rating factor of 10-30%.

In a second mode, the user may select a GUI setting to enable afree-floating bit-rate and a fixed bit-error-rate. Based on thespecified maximum bit-error-rate, the real-time bit-error-rate analyzer406 detects the error-rate and the bit-rate is automatically andcontinuously adjusted, incremented to decremented, to hold thebit-error-rate at the specified maximum bit-error-rate. The user mayselect the maximum bit-error-rate from a menu of typical values in arange from ideal to poor.

In some implementations or embodiments, the fixed bit-error-rate may bedynamically determined, for example in response to demand, so thathigher error rates may be tolerated if the demand is uniquely met byother aspects of the system. For example, a system may have additionalerror detection capability to reduce or minimize the number ofundetected errors. Internet Small Computer Systems Interface (iSCSI) hasa suitable error detection capability. Demand may be determined bysensing of various conditions such as throughput, queuing, and the like.

During typical operation, once a user makes selections via the GUI 408,and selects possible de-rating factors such as 10%, 20%, or the like,the throughput control maintains a substantially steady-state with nochange to initialized parameters unless a physical change affects thelink, such as a degraded or broken physical connection, a cable bendradius is altered, or the like. In the event the standard protocol, forexample Fibre Channel protocol, encounters error and re-tries, thethroughput enhancement controller-enabled devices, for example host busadapter, may be enabled to make automatic adjustments until the re-triesno longer occur.

A typical apparatus implementation may include the bit-error-rateanalyzer 406 and other link analysis capabilities, for example bygenerating continuous data or control characters on a transmit path andexpect continuous data or control characters on the receive path.Monitoring and analysis of the received data enables link faultdetection and generation of signals indicating reliability of the link.The apparatus 400 may implement test pattern generation and checking forlink testing and system testing. The bit-error-rate analyzer 406typically performs monitoring and generates an error rate indication toenable constant line quality monitoring.

In a typical embodiment, the bit-error-rate analyzer 406 is transparentto the link and generates bit error rate patterns using AmericanNational Standards Institute (ANSI) standards. The patterns are framedusing suitable framing protocols so that the link is tested with actualdata traffic at actual operating speeds of the data link. Bit errors arecounted and used to supply a statistical measurement of data linkquality.

In a particular implementation, a pair of Fibre Channel interface cardsmay be equipped with throughput enhancement devices and used tofacilitate communications within a single electronics rack, for examplewith an approximately one meter cable, to operate at a data rate of fourto eighty times the standard 1-2 Gb throughput rate. A similarconnection within a single data center, for example with anapproximately ten meter cable, may operate at a data rate of four to tentimes throughput improvement. Even a link with a multiple-kilometerlength dark fiber, for example a premium optical fiber in a privatetrench, may realize a two times or more improvement in industry-standardthroughput.

Throughput of long distance replication links is generally the singlemost limiting factor—the weakest link—in a multiple million dollar datareplication arrangement. Accordingly, implementation of the illustrativethroughput enhancement may dramatically improve the total number oflinks implemented in a system, the initial copy times for data volumes,and the maximum traffic per link. A reduction in the total number ofimplemented links may save substantial expense which, over time, mayeclipse the cost of the storage arrays in the replication system.

For short-distance links, threshold enhancement-equipped Fiber Channelinterface cards may be used as an alternative to infini-band and otherhigh bandwidth link types.

While the present disclosure describes various embodiments, theseembodiments are to be understood as illustrative and do not limit theclaim scope. Many variations, modifications, additions and improvementsof the described embodiments are possible. For example, those havingordinary skill in the art will readily implement the steps necessary toprovide the structures and methods disclosed herein, and will understandthat the process parameters, materials, and dimensions are given by wayof example only. The parameters, materials, and dimensions can be variedto achieve the desired structure as well as modifications, which arewithin the scope of the claims. Variations and modifications of theembodiments disclosed herein may also be made while remaining within thescope of the following claims. For example, the disclosed apparatus andtechnique can be used in any storage and communication configurationwith any appropriate number of storage arrays or elements. The variouscommunication controller, analyzer, and bit test elements may beimplemented in any suitable component or device, for example hostcomputers, host bus adapters, storage controllers, disk controllers,management appliances, and the like. The links may be in anyconfiguration and have lengths of any suitable distance. Although, theillustrative system discloses magnetic disk storage elements, anyappropriate type of storage technology may be implemented. Controlelements may be implemented as software or firmware on general purposecomputer systems, workstations, servers, and the like, but may beotherwise implemented on special-purpose devices and embedded systems.

1. A data-replication storage system that minors information on two ormore storage devices, the system comprising: a storage array; and acommunication controller that is part of an electronic device and thatcommunicates information to be stored in the storage array, on anindustry standard link which is compliant with a published standard andon a private link comprising a fiber link operated in deviation from thepublished standard at a throughput higher than maximum throughput of thepublished standard, the communication controller monitors private linkperformance and selectively increases information throughput on theprivate link above the maximum standard throughput by an amountdetermined by the monitored private link performance.
 2. The systemaccording to claim 1 further comprising: the communication controllerthat communicates information on the private link using a communicationtechnique selected from among multiple-tiered, progressively higherthroughput communication techniques operable on the private link, andwherein the electronic device comprises a host bus adapter in thedata-replication storage system; and a graphical user interface coupledto the communication controller that enables a user to select from amongthe plurality of communication techniques.
 3. The system according toclaim 1 further comprising: a bit error rate analyzer coupled to thecommunication controller, wherein the communication controllercommunicates information on the private link using a communicationtechnique that selectively increases clock rate throughput rate andmutually optimizes bit rate and bit error rate, and wherein theelectronic device is disposed in the data-replication storage system. 4.The system according to claim 1 further comprising: the communicationcontroller that communicates information on the private link using adigital pulse width modulation communication technique that encodesinformation as an increment in pulse width over a nominal pulse width.5. The system according to claim 1 further comprising: a bit error rateanalyzer coupled to the communication controller, wherein thecommunication controller communicates information on the private linkusing a communication technique that selectively increases clock ratethroughput rate and mutually optimizes bit rate and bit error rate andencodes information as an increment in pulse width over a nominal pulsewidth in a digital pulse width modulation communication technique. 6.The system according to claim 1 further comprising: a bit error rateanalyzer coupled to the communication controller, wherein thecommunication controller communicates information on the private linkusing a communication technique that selectively increases clock ratethroughput rate and mutually optimizes bit rate and bit error rate,encodes information as an increment in pulse width over a nominal pulsewidth in a digital pulse width modulation communication technique, andselects a signal data compaction level based on the bit error rate.
 7. Astorage apparatus comprising: a storage array; a bit error rateanalyzer; and an electronic device comprising a communication controllercoupled to the bit error rate analyzer and the storage array, whereinthe communication controller communicates information to be stored inthe storage array on an industry standard link which is compliant with apublished standard and on a private link operated in deviation from thepublished standard at a throughput higher than maximum throughput of thepublished standard,. wherein the communication controller monitorsprivate link performance and selectively increases informationthroughput on the private link above the maximum standard throughput byan amount determined by the monitored private link performance, andcommunicates information on the private link using a communicationtechnique that selectively increases clock rate throughput rate andmutually optimizes bit rate and bit error rate.
 8. The apparatusaccording to claim 7 further comprising: a graphical user interfacecoupled to the communication controller that enables the user toselectively operate the communication controller in a first mode thatenables a free-floating bit-error rate, a fixed bit rate, andcommunication at a highest bit rate at which an acceptable bit-errorrate is attained, and to selectively operate in a second mode thatenables a free-floating bit rate, a fixed bit-error rate, andcommunication that adjusts bit rate to a maximum, and wherein theelectronic device comprising the communication controller is a componentof a geographically-dispersed data storage system.
 9. The apparatusaccording to claim 7 wherein: the communication controller communicatesinformation using a communication technique that selectively increasesclock rate throughput rate and mutually optimizes bit rate and bit errorrate and encodes information as an increment in pulse width over anominal pulse width in a digital pulse width modulation communicationtechnique.
 10. The apparatus according to claim 7 wherein: thecommunication controller communicates information on the private linkusing a communication technique that selectively increases clock ratethroughput rate and mutually optimizes bit rate and bit error rate,encodes information as an increment in pulse width over a nominal pulsewidth in a digital pulse width modulation communication technique, andselects a signal data compaction level based on the bit error rate. 11.A communication method of increasing throughput in a data storagesystem, the method comprising: communicating, via an electronic devicecomprising a communication controller in the data storage system,information to be stored in a storage array on an industry standard linkwhich is compliant with a published standard; communicating, via thecommunication controller, information to be stored in the storage arrayon a private link comprising a fiber link operated in deviation from thepublished standard at a throughput higher than maximum throughput of thepublished standard; monitoring private link performance; and increasinginformation throughput on the private link above the maximum standardthroughput by an amount determined by the monitored private linkperformance.
 12. The method according to claim 11 further comprising:monitoring bit error rate on the private link; selectively increasingclock rate throughput rate; selectively enabling a free-floatingbit-error rate, a fixed bit rate, and communication at a highest bitrate at which an acceptable bit-error rate is attained; and selectivelyenabling a free-floating bit rate, a fixed bit-error rate, andcommunication that adjusts bit rate to a maximum.
 13. The methodaccording to claim 11 further comprising: selectively increasing clockrate throughput rate; mutually optimizing bit rate and bit error rate;and encoding information as an increment in pulse width over a nominalpulse width in a digital pulse width modulation communication technique.14. The method according to claim 11 further comprising: selectivelyincreasing clock rate throughput rate; mutually optimizing bit rate andbit error rate; encoding information as an increment in pulse width overa nominal pulse width in a digital pulse width modulation communicationtechnique; and selecting a signal data compaction level based on the biterror rate.
 15. The system according to claim 1, wherein no terminus ofthe private link is directly attached to a public, shared network thatsupports a standard managed network, and wherein the private linkcomprises a communication link that is exclusive of any industrystandard links that operate primarily at an industry standard throughputrate.
 16. The system according to claim 1, wherein the electronic devicehas an Ethernet interface and a switch fabric interconnect device. 17.The apparatus according to claim 7, wherein the electronic devicecomprising the communication controller comprises a host bus adapter, astorage controller, or a disk controller, or any combination thereof.18. The apparatus according to claim 7, wherein the private linkcomprises a fiber optic cable and is configured to only communicateinformation associated with a company leasing or owning the privatelink.
 19. The method according to claim 11, wherein communicatinginformation on the private link comprises only communicating data of acompany leasing or owning the private link, and disregarding the datarate constraints of the published standard.
 20. The method according toclaim 11, wherein communicating information on the private linkcomprises communicating data to be stored in the storage array from ahost via the electronic device comprising a host bus adapter or storagecontroller, and following on the private link protocols similar oridentical to protocols of the published standard associated with theindustry standard link.